Redesign the API home page as a navigable table directory (LMFDB#4964) - #40
Redesign the API home page as a navigable table directory (LMFDB#4964)#40roed-math wants to merge 3 commits into
Conversation
Group the tables on /api into datasets with human-readable names (so hgcwa etc. are explained), show row counts (free: in-memory meta_tables totals) and short table descriptions (one bulk query of the tables.<name> knowls via a new knowldb.get_table_descriptions), add a client-side filter box and a jump-to-dataset strip, collapse the long usage docs into a <details> block, link the stats and access options pages, and actually hide test tables by default (the show_hidden flag and /api/all were dead code). All existing URLs and machine formats are unchanged. Verified with the flask test client (page loads in ~0.16s warm; all 193 table links resolve; JSON/YAML output unchanged), in a live browser on port 38046 (filter, no-match state and details work with no console errors), sage -python -m pytest lmfdb/api/test_api.py (7 passed) and pyflakes. git merge-tree confirms no conflicts with open PRs #17/#18/#22. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hide the test tables that use the *_test naming convention as well as those named test_*, so the default directory really omits them; both conventions are covered by a regression test, and the page copy now says "test and auxiliary tables" rather than "test tables", since hidden_collection() also matches auxiliary names. Call the fq dataset "Finite fields": fq_fields stores finite fields (characteristic, degree, conway, polynomial), not function fields. Return the current revision of a description knowl. The new bulk lookup and the existing get_table_description/get_column_descriptions all combined DISTINCT ON (id) with an ascending timestamp, which picks the revision a description was written with rather than the latest one: mf_hecke_cc displayed "Hecke eignvalues", ec_nfcurves an outdated description, and gps_subgroup_search the "not yet updated on this server" placeholder. The three now share a _description_selecter helper that sorts revisions newest first, as get_knowl does, so they cannot drift; which statuses are visible is unchanged. Log the exception when the bulk description query fails instead of silently emptying a column of the page, and add a test that a known description reaches both the table row and the filter metadata. Verified with sage -python -m pytest on lmfdb/api/test_api.py, lmfdb/tests/test_dynamic_knowls.py and lmfdb/tests/test_connection_reset.py (22 passed, 1 skipped); every added assertion was checked to fail with its fix reverted; pyflakes, pylint and ruff clean; /api/ still renders in 0.19s warm and the filter still matches names, topics and descriptions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks for the review. All four points are addressed in db63a96; nothing else about the page changed. 1. Test tables using the
|
| table | first revision (what was displayed) | current revision (displayed now) |
|---|---|---|
mf_hecke_cc |
Hecke eignvalues of embedded newforms | Hecke eigenvalues of embedded newforms |
ec_nfcurves |
Elliptic curves over *Q* and other number fields |
Elliptic curves over number fields other than $\Q$ |
gps_subgroup_search |
(description not yet updated on this server) | Subgroup search data for finite groups |
get_table_descriptions(), get_table_description() and get_column_descriptions() now share a _description_selecter(fields, match) helper that orders id, timestamp DESC, so the three cannot drift apart. The helper's docstring says why the ordering is what makes DISTINCT ON (id) return the current revision. Visibility is deliberately unchanged: the queries still select type = 2 AND status >= 0, so this only changes which of the eligible revisions wins. get_table_description() lost its trailing LIMIT 1, which was a no-op once the id is matched with =.
Regression test DescriptionKnowlTest (in lmfdb/api/test_api.py): it looks for description knowls with more than one distinct content, then asserts that the single, bulk and column lookups each return the content of the newest revision, taken independently from get_edit_history(...)[-1]. Devmirror has 7 such table knowls and 126 such column knowls, so it is not vacuous, and I checked that it fails when DESC is removed. I kept it in test_api.py rather than starting a knowl test module because CI runs an explicit list of test files and asserts that the repository contains exactly 44 of them; happy to move it and update generate-matrix.py/matrix_includes.json if you would prefer that.
3. fq
Now "Finite fields". test_api_home asserts the rendered heading fq — Finite fields.
4. Observability
The bulk-lookup fallback now logs with logger.exception("Could not load the table descriptions for the API index") before falling back to empty descriptions. New test_api_home_descriptions patches knowldb.get_table_descriptions with a test-owned string and asserts it appears both in the row's description cell and in the data-desc attribute the filter searches; it then checks the unpatched page still renders a substantial number of non-empty descriptions, so a broken query or import cannot leave CI green.
Verification
sage -python -m pytest lmfdb/api/test_api.py lmfdb/tests/test_dynamic_knowls.py lmfdb/tests/test_connection_reset.py
22 passed, 1 skipped
Each new assertion was checked to fail with its fix reverted (4 failures: the two ordering tests, the fq heading, the _test tables). pyflakes, pylint and ruff check --preview --select=E722 are clean on the changed files. /api/ still renders in 0.19s warm with one bulk knowl query, and filtering the rendered page still matches table names, dataset topics and descriptions, with all 184 visible rows restored when the box is cleared.
🤖 Generated with Claude Code
|
GPT signed off. |
|
Superseded by LMFDB#7174, opened upstream from this same branch. Closing here; review continues upstream. |
2 participants
The /api index was a flat dump of hundreds of table names. This PR turns it into a
directory: tables are grouped into datasets with human-readable topic names (so it is clear
what e.g. hgcwa stands for), each table shows its row count and its short description (the
tables.<name>knowl), and a small client-side filter box plus a jump-to-dataset strip makethe page searchable (it degrades gracefully without JS). The long query-syntax documentation
is collapsed into a details block, the stats and access-options pages are linked from the
top and the sidebar, and test tables are now really hidden by default —
/api/all(whoseshow_hidden flag was previously dead code) shows them. The page stays server-rendered and
fast: row counts are the in-memory meta_tables totals and all descriptions come from a
single bulk knowl query, so there are still no per-table queries at page load; all existing
URLs and the JSON/YAML output are unchanged. Verified with the api test suite (7 passed,
including two new tests), timing and link checks through the flask test client, and a live
browser check of the filter;
git merge-treeconfirms it merges cleanly with theoutstanding api PRs from this series. Addresses LMFDB#4964.
🤖 Generated with Claude Code